Bi-Level Adaptive Storage Expansion Strategy for Microgrids Using Deep Reinforcement Learning

Battery energy storage (BES) is a versatile resource for the secure and economic operation of microgrids (MGs). Prevailing stochastic optimization-based approaches BES expansion planning MGs are computationally complicated. This work proposes data-driven bi-level multi-period framework to determine siting, sizing, timing installations. The proposed unifies deep reinforcement learning (DRL) linear programming, thereby decoupling determinations integer continuous decision variables in two time scales, respectively. In upper level, rainbow DRL agent with quantile regression trained provide dynamic policies accommodate renewable resources (RESs), load, battery price changes efficiently. lower level computes optimal frequency constraints hedge islanding contingency. levels communicate one another by exchanging configuration operating expenses order accomplish shared goal minimizing investment costs. Comparative case studies on an MG carried out demonstrate superiority DRL-based solution mixed-integer programming counterpart efficiency, scalability, adaptability.